A Human Systems Integration Method for Validating Team

Improving simulation-based training is the focus of a Navy program of research titled Debriefing Distributed Scenario-Based Exercises (DDSBE). DDSBE is intended to optimize training effectiveness by demonstrating improved methods and technologies for assessing team and multi-team performance, diagnosis routines, and debriefing templates. A main objective of the DDSBE program in spiral 1 was to test the training simulator capability to produce valid and reliable team performance assessments. This paper describes a human systems integration test methodology and an exploratory experiment that applied the methodology. The results showed excellent assessment validity and inter-rater reliability when a handheld PC was used to assess team task and teamwork behaviors in conjunction with an automated performance recording capability. The results of this analysis should provide guidance to simulation developers in designing distributed training that supports learning combat team skills.